{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regressão Linear Multivariada - Trabalho\n",
    "\n",
    "## Estudo de caso: Qualidade de Vinhos\n",
    "\n",
    "Nesta trabalho, treinaremos um modelo de regressão linear usando descendência de gradiente estocástico no conjunto de dados da Qualidade do Vinho. O exemplo pressupõe que uma cópia CSV do conjunto de dados está no diretório de trabalho atual com o nome do arquivo *winequality-white.csv*.\n",
    "\n",
    "O conjunto de dados de qualidade do vinho envolve a previsão da qualidade dos vinhos brancos em uma escala, com medidas químicas de cada vinho. É um problema de classificação multiclasse, mas também pode ser enquadrado como um problema de regressão. O número de observações para cada classe não é equilibrado. Existem 4.898 observações com 11 variáveis de entrada e 1 variável de saída. Os nomes das variáveis são os seguintes:\n",
    "\n",
    "1. Fixed acidity.\n",
    "2. Volatile acidity.\n",
    "3. Citric acid.\n",
    "4. Residual sugar.\n",
    "5. Chlorides.\n",
    "6. Free sulfur dioxide. \n",
    "7. Total sulfur dioxide. \n",
    "8. Density.\n",
    "9. pH.\n",
    "10. Sulphates.\n",
    "11. Alcohol.\n",
    "12. Quality (score between 0 and 10).\n",
    "\n",
    "O desempenho de referencia de predição do valor médio é um RMSE de aproximadamente 0.148 pontos de qualidade.\n",
    "\n",
    "Utilize o exemplo apresentado no tutorial e altere-o de forma a carregar os dados e analisar a acurácia de sua solução. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Definição das Bibliotecas e Funções Principais"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def RMSE(errors):\n",
    "    return np.sqrt(1/errors.shape[1] * np.sum(errors**2))\n",
    "\n",
    "def predict(X, coef, addOnes=False):\n",
    "    if(addOnes): X = np.append(np.ones([X.shape[0], 1]), X, axis=1)\n",
    "    return np.dot(X, coef).reshape(1, X.shape[0])\n",
    "\n",
    "def stochasticGD(X, y, alfa=0.00001, maxEpoch=50):\n",
    "    X = np.append(np.ones([X.shape[0], 1]), X, axis=1)\n",
    "    coef = np.random.randn(X.shape[1], 1)\n",
    "    errorHist = []\n",
    "    \n",
    "    for epoch in range(maxEpoch):\n",
    "        error =  predict(X, coef) - y\n",
    "        errorHist.append(RMSE(error))\n",
    "        \n",
    "        for i in range(X.shape[0]):\n",
    "            coef[0] -= alfa * error[0,i]\n",
    "            for j in range(len(coef)-1):\n",
    "                coef[j+1] -= alfa * error[0,i] * X[i,j]\n",
    "                \n",
    "        print(\"Epoch: {} | RMSE: {}\".format(epoch, errorHist[-1]))\n",
    "        print(\"Coefficients: \\n\", coef.T)\n",
    "        print(\"\\n###\")\n",
    "    \n",
    "    return coef, errorHist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Carregando o conjunto de dados e utilizando o Gradiente Descendente Estocástico"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"winequality-white.csv\", delimiter=\";\")\n",
    "\n",
    "X = MinMaxScaler().fit_transform(data.values[:,:-1])\n",
    "y = data.values[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0 | RMSE: 5.603006430346713\n",
      "Coefficients: \n",
      " [[ 1.41482737 -0.14446488 -1.36397147 -1.72856557  0.25729327  0.29945667\n",
      "  -1.72824681 -0.14588103  1.0290101   0.72245113 -0.25032003 -0.22558473]]\n",
      "\n",
      "###\n",
      "Epoch: 1 | RMSE: 5.116385508659133\n",
      "Coefficients: \n",
      " [[ 1.66132821  0.10203597 -1.29172991 -1.68097011  0.30732737  0.32096455\n",
      "  -1.70195755 -0.11703484  1.10247275  0.75444081 -0.14551486 -0.14777653]]\n",
      "\n",
      "###\n",
      "Epoch: 2 | RMSE: 4.674617386194008\n",
      "Coefficients: \n",
      " [[ 1.88584284  0.3265506  -1.22602007 -1.63768232  0.35290373  0.34050594\n",
      "  -1.67806477 -0.09074454  1.16930883  0.78348593 -0.05005471 -0.07696228]]\n",
      "\n",
      "###\n",
      "Epoch: 3 | RMSE: 4.273807335226205\n",
      "Coefficients: \n",
      " [[ 2.09033743  0.53104519 -1.16625786 -1.59831694  0.394421    0.35825667\n",
      "  -1.65635417 -0.06678155  1.23011095  0.8098498   0.03689623 -0.01251654]]\n",
      "\n",
      "###\n",
      "Epoch: 4 | RMSE: 3.9104159314646747\n",
      "Coefficients: \n",
      " [[ 2.27660233  0.71731009 -1.11191145 -1.56252317  0.43224216  0.37437687\n",
      "  -1.63663059 -0.04493775  1.2854187   0.83377217  0.11609899  0.0461302 ]]\n",
      "\n",
      "###\n",
      "Epoch: 5 | RMSE: 3.5812267374506135\n",
      "Coefficients: \n",
      " [[ 2.44626777  0.88697552 -1.06249654 -1.52998157  0.46669774  0.38901235\n",
      "  -1.61871634 -0.02502363  1.33572341  0.85547139  0.18824656  0.09949652]]\n",
      "\n",
      "###\n",
      "Epoch: 6 | RMSE: 3.283316528662896\n",
      "Coefficients: \n",
      " [[ 2.60081819  1.04152595 -1.01757216 -1.50040128  0.49808872  0.40229585\n",
      "  -1.60244961 -0.00686662  1.38147248  0.87514623  0.25396995  0.14805461]]\n",
      "\n",
      "###\n",
      "Epoch: 7 | RMSE: 3.014027756319595\n",
      "Coefficients: \n",
      " [[ 2.7416053   1.18231306 -0.97673677 -1.47351744  0.52668916  0.41434827\n",
      "  -1.58768308  0.00969042  1.4230733   0.89297773  0.31384375  0.19223447]]\n",
      "\n",
      "###\n",
      "Epoch: 8 | RMSE: 2.770942967636425\n",
      "Coefficients: \n",
      " [[ 2.86985991  1.31056767 -0.93962473 -1.44908889  0.5527486   0.4252797\n",
      "  -1.57428257  0.02479056  1.46089684  0.90913071  0.36839114  0.2324276 ]]\n",
      "\n",
      "###\n",
      "Epoch: 9 | RMSE: 2.5518609377083132\n",
      "Coefficients: \n",
      " [[ 2.98670277  1.42741053 -0.90590312 -1.42689603  0.57649428  0.43519036\n",
      "  -1.56212593  0.03856409  1.49528087  0.92375527  0.41808855  0.26899051]]\n",
      "\n",
      "###\n",
      "Epoch: 10 | RMSE: 2.3547743108144545\n",
      "Coefficients: \n",
      " [[ 3.0931544   1.53386216 -0.87526877 -1.40673889  0.5981331   0.44417153\n",
      "  -1.55110192  0.05112965  1.52653297  0.93698807  0.46336978  0.30224785]]\n",
      "\n",
      "###\n",
      "Epoch: 11 | RMSE: 2.177848606124745\n",
      "Coefficients: \n",
      " [[ 3.19014406  1.63085181 -0.84744563 -1.38843539  0.61785348  0.45230632\n",
      "  -1.54110925  0.06259526  1.55493321  0.94895356  0.50462984  0.3324952 ]]\n",
      "\n",
      "###\n",
      "Epoch: 12 | RMSE: 2.019402515391679\n",
      "Coefficients: \n",
      " [[ 3.2785179   1.71922566 -0.82218235 -1.37181973  0.63582697  0.4596704\n",
      "  -1.53205569  0.07305929  1.58073661  0.95976506  0.54222842  0.36000172]]\n",
      "\n",
      "###\n",
      "Epoch: 13 | RMSE: 1.8778895070448227\n",
      "Coefficients: \n",
      " [[ 3.35904641  1.79975417 -0.79925003 -1.35674094  0.65220979  0.46633268\n",
      "  -1.52385725  0.0826113   1.6041754   0.96952575  0.57649303  0.38501252]]\n",
      "\n",
      "###\n",
      "Epoch: 14 | RMSE: 1.7518808459940758\n",
      "Coefficients: \n",
      " [[ 3.43243119  1.87313894 -0.77844026 -1.34306156  0.66714418  0.47235592\n",
      "  -1.51643747  0.09133286  1.62546102  0.9783296   0.60772192  0.40775075]]\n",
      "\n",
      "###\n",
      "Epoch: 15 | RMSE: 1.640050228816099\n",
      "Coefficients: \n",
      " [[ 3.49931106  1.94001882 -0.75956326 -1.3306564   0.68075968  0.47779723\n",
      "  -1.50972671  0.09929823  1.64478602  0.98626215  0.63618663  0.42841964]]\n",
      "\n",
      "###\n",
      "Epoch: 16 | RMSE: 1.541160300968715\n",
      "Coefficients: \n",
      " [[ 3.56026774  2.0009755  -0.74244623 -1.31941149  0.69317422  0.48270866\n",
      "  -1.50366155  0.10657503  1.66232572  0.9934013   0.66213445  0.44720423]]\n",
      "\n",
      "###\n",
      "Epoch: 17 | RMSE: 1.4540513439012783\n",
      "Coefficients: \n",
      " [[ 3.61583093  2.05653868 -0.72693182 -1.30922303  0.7044952   0.48713758\n",
      "  -1.49818428  0.11322485  1.67823976  0.999818    0.68579054  0.46427302]]\n",
      "\n",
      "###\n",
      "Epoch: 18 | RMSE: 1.3776323756560345\n",
      "Coefficients: \n",
      " [[ 3.66648294  2.10719069 -0.71287674 -1.29999652  0.71482041  0.49112711\n",
      "  -1.49324231  0.11930374  1.69267351  1.00557683  0.70735993  0.47977943]]\n",
      "\n",
      "###\n",
      "Epoch: 19 | RMSE: 1.3108747899419195\n",
      "Coefficients: \n",
      " [[ 3.71266296  2.15337072 -0.70015052 -1.2916459   0.7242389   0.49471654\n",
      "  -1.48878776  0.12486276  1.70575934  1.01073662  0.7270293   0.49386316]]\n",
      "\n",
      "###\n",
      "Epoch: 20 | RMSE: 1.2528084752887587\n",
      "Coefficients: \n",
      " [[ 3.75477091  2.19547867 -0.68863438 -1.28409281  0.73283174  0.49794164\n",
      "  -1.48477706  0.12994841  1.71761778  1.01535092  0.74496865  0.50665143]]\n",
      "\n",
      "###\n",
      "Epoch: 21 | RMSE: 1.2025201383098463\n",
      "Coefficients: \n",
      " [[ 3.79317094  2.2338787  -0.67822012 -1.27726591  0.74067277  0.50083498\n",
      "  -1.48117051  0.13460302  1.72835858  1.01946849  0.76133278  0.51826007]]\n",
      "\n",
      "###\n",
      "Epoch: 22 | RMSE: 1.1591533492785853\n",
      "Coefficients: \n",
      " [[ 3.82819463  2.26890239 -0.66880925 -1.27110021  0.74782922  0.50342621\n",
      "  -1.47793196  0.13886512  1.73808168  1.02313375  0.77626264  0.52879457]]\n",
      "\n",
      "###\n",
      "Epoch: 23 | RMSE: 1.1219096823484431\n",
      "Coefficients: \n",
      " [[ 3.86014391  2.30085167 -0.66031208 -1.26553655  0.7543623   0.50574233\n",
      "  -1.4750285   0.14276984  1.74687807  1.02638712  0.78988657  0.53835096]]\n",
      "\n",
      "###\n",
      "Epoch: 24 | RMSE: 1.0900502712824562\n",
      "Coefficients: \n",
      " [[ 3.88929372  2.33000148 -0.65264694 -1.26052108  0.76032776  0.50780793\n",
      "  -1.47243017  0.14634911  1.75483062  1.02926543  0.80232145  0.54701669]]\n",
      "\n",
      "###\n",
      "Epoch: 25 | RMSE: 1.0628971537160334\n",
      "Coefficients: \n",
      " [[ 3.91589439  2.35660215 -0.64573945 -1.25600472  0.76577636  0.5096454\n",
      "  -1.47010968  0.14963205  1.76201478  1.03180221  0.81367368  0.55487142]]\n",
      "\n",
      "###\n",
      "Epoch: 26 | RMSE: 1.039833914857127\n",
      "Coefficients: \n",
      " [[ 3.94017389  2.38088165 -0.63952189 -1.25194283  0.77075431  0.51127514\n",
      "  -1.46804218  0.15264515  1.76849926  1.03402797  0.82404017  0.56198763]]\n",
      "\n",
      "###\n",
      "Epoch: 27 | RMSE: 1.020305328278051\n",
      "Coefficients: \n",
      " [[ 3.96233979  2.40304755 -0.63393262 -1.24829472  0.77530369  0.5127157\n",
      "  -1.46620505  0.15541255  1.77434661  1.03597052  0.83350916  0.56843137]]\n",
      "\n",
      "###\n",
      "Epoch: 28 | RMSE: 1.0038158848535903\n",
      "Coefficients: \n",
      " [[ 3.98258109  2.42328885 -0.62891547 -1.24502335  0.77946284  0.51398398\n",
      "  -1.46457768  0.15795621  1.7796138   1.03765518  0.84216099  0.57426276]]\n",
      "\n",
      "###\n",
      "Epoch: 29 | RMSE: 0.989927266047197\n",
      "Coefficients: \n",
      " [[ 4.0010699   2.44177766 -0.62441932 -1.24209502  0.78326663  0.51509539\n",
      "  -1.46314132  0.16029615  1.78435269  1.03910498  0.85006881  0.57953654]]\n",
      "\n",
      "###\n",
      "Epoch: 30 | RMSE: 0.9782549343084114\n",
      "Coefficients: \n",
      " [[ 4.01796293  2.45867069 -0.62039763 -1.239479    0.78674686  0.51606393\n",
      "  -1.46187888  0.16245058  1.78861053  1.04034092  0.85729925  0.58430256]]\n",
      "\n",
      "###\n",
      "Epoch: 31 | RMSE: 0.9684640763744312\n",
      "Coefficients: \n",
      " [[ 4.03340289  2.47411065 -0.61680799 -1.23714734  0.78993244  0.51690238\n",
      "  -1.4607748   0.16443609  1.79243032  1.0413821   0.86391296  0.58860621]]\n",
      "\n",
      "###\n",
      "Epoch: 32 | RMSE: 0.9602651514179553\n",
      "Coefficients: \n",
      " [[ 4.04751971  2.48822747 -0.6136118  -1.23507458  0.79284973  0.51762234\n",
      "  -1.45981494  0.16626779  1.79585123  1.04224594  0.86996518  0.59248883]]\n",
      "\n",
      "###\n",
      "Epoch: 33 | RMSE: 0.9534092779954162\n",
      "Coefficients: \n",
      " [[ 4.06043171  2.50113947 -0.61077393 -1.23323753  0.79552271  0.5182344\n",
      "  -1.45898638  0.16795942  1.79890891  1.04294826  0.8755062   0.59598806]]\n",
      "\n",
      "###\n",
      "Epoch: 34 | RMSE: 0.9476836555510322\n",
      "Coefficients: \n",
      " [[ 4.07224663  2.51295439 -0.60826235 -1.23161509  0.79797323  0.51874821\n",
      "  -1.45827738  0.16952352  1.80163583  1.04350351  0.88058183  0.59913816]]\n",
      "\n",
      "###\n",
      "Epoch: 35 | RMSE: 0.9429071697568481\n",
      "Coefficients: \n",
      " [[ 4.08306258  2.52377034 -0.60604794 -1.23018802  0.80022117  0.51917252\n",
      "  -1.45767726  0.17097149  1.80406156  1.04392482  0.88523375  0.60197034]]\n",
      "\n",
      "###\n",
      "Epoch: 36 | RMSE: 0.938926284782838\n",
      "Coefficients: \n",
      " [[ 4.09296887  2.53367663 -0.60410416 -1.22893883  0.80228465  0.51951533\n",
      "  -1.45717627  0.17231371  1.80621303  1.04422415  0.88949996  0.60451302]]\n",
      "\n",
      "###\n",
      "Epoch: 37 | RMSE: 0.9356112848279582\n",
      "Coefficients: \n",
      " [[ 4.10204686  2.54275462 -0.60240683 -1.22785159  0.80418017  0.51978393\n",
      "  -1.45676555  0.17355965  1.80811475  1.0444124   0.89341501  0.60679207]]\n",
      "\n",
      "###\n",
      "Epoch: 38 | RMSE: 0.93285289424897\n",
      "Coefficients: \n",
      " [[ 4.11037061  2.55107837 -0.60093396 -1.22691176  0.80592273  0.51998492\n",
      "  -1.45643703  0.1747179   1.80978904  1.04449948  0.8970104   0.60883105]]\n",
      "\n",
      "###\n",
      "Epoch: 39 | RMSE: 0.9305592808160877\n",
      "Coefficients: \n",
      " [[ 4.11800756  2.55871532 -0.59966552 -1.22610615  0.80752601  0.52012434\n",
      "  -1.45618335  0.17579631  1.81125624  1.04449443  0.9003148   0.6106514 ]]\n",
      "\n",
      "###\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 40 | RMSE: 0.9286534292870214\n",
      "Coefficients: \n",
      " [[ 4.12501913  2.56572689 -0.59858325 -1.22542271  0.80900247  0.52020768\n",
      "  -1.45599782  0.17680202  1.81253484  1.04440547  0.90335431  0.61227268]]\n",
      "\n",
      "###\n",
      "Epoch: 41 | RMSE: 0.9270708613758998\n",
      "Coefficients: \n",
      " [[ 4.13146123  2.57216899 -0.59767056 -1.22485049  0.81036344  0.52023995\n",
      "  -1.45587435  0.17774153  1.8136417   1.04424009  0.9061527   0.61371266]]\n",
      "\n",
      "###\n",
      "Epoch: 42 | RMSE: 0.9257576719188847\n",
      "Coefficients: \n",
      " [[ 4.13738479  2.57809255 -0.5969123  -1.22437952  0.81161924  0.5202257\n",
      "  -1.45580738  0.17862075  1.81459217  1.04400511  0.90873162  0.61498754]]\n",
      "\n",
      "###\n",
      "Epoch: 43 | RMSE: 0.9246688483158175\n",
      "Coefficients: \n",
      " [[ 4.14283617  2.58354392 -0.59629471 -1.2240007   0.81277927  0.52016908\n",
      "  -1.45579186  0.17944509  1.81540023  1.04370673  0.91111077  0.61611207]]\n",
      "\n",
      "###\n",
      "Epoch: 44 | RMSE: 0.9237668400669623\n",
      "Coefficients: \n",
      " [[ 4.14785758  2.58856534 -0.59580525 -1.22370576  0.8138521   0.52007386\n",
      "  -1.45582319  0.18021944  1.81607859  1.04335062  0.9133081   0.61709968]]\n",
      "\n",
      "###\n",
      "Epoch: 45 | RMSE: 0.9230203465641086\n",
      "Coefficients: \n",
      " [[ 4.15248749  2.59319524 -0.5954325  -1.22348718  0.81484552  0.51994348\n",
      "  -1.45589717  0.18094828  1.81663885  1.04294192  0.91533996  0.6179626 ]]\n",
      "\n",
      "###\n",
      "Epoch: 46 | RMSE: 0.9224032935884039\n",
      "Coefficients: \n",
      " [[ 4.15676088  2.59746863 -0.59516605 -1.22333808  0.81576663  0.51978106\n",
      "  -1.45601     0.18163567  1.81709156  1.04248532  0.91722123  0.61871195]]\n",
      "\n",
      "###\n",
      "Epoch: 47 | RMSE: 0.9218939717499177\n",
      "Coefficients: \n",
      " [[ 4.16070963  2.60141739 -0.59499644 -1.22325223  0.81662188  0.51958948\n",
      "  -1.4561582   0.18228532  1.81744631  1.04198509  0.91896545  0.61935789]]\n",
      "\n",
      "###\n",
      "Epoch: 48 | RMSE: 0.9214743130584504\n",
      "Coefficients: \n",
      " [[ 4.16436278  2.60507054 -0.59491505 -1.22322394  0.81741717  0.5193713\n",
      "  -1.4563386   0.18290061  1.81771186  1.04144511  0.92058496  0.61990964]]\n",
      "\n",
      "###\n",
      "Epoch: 49 | RMSE: 0.9211292847340766\n",
      "Coefficients: \n",
      " [[ 4.16774675  2.6084545  -0.59491402 -1.22324803  0.81815785  0.51912891\n",
      "  -1.45654833  0.1834846   1.81789618  1.04086892  0.922091    0.62037561]]\n",
      "\n",
      "###\n"
     ]
    }
   ],
   "source": [
    "[coef, errorHist] = stochasticGD(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gradiente Descendente Estocástico\n",
      "RMSE: 0.9208463821240235\n",
      "Coeficientes:\n",
      " [[ 4.16774675  2.6084545  -0.59491402 -1.22324803  0.81815785  0.51912891\n",
      "  -1.45654833  0.1834846   1.81789618  1.04086892  0.922091    0.62037561]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Gradiente Descendente Estocástico\\nRMSE: {}\".format(RMSE(y - predict(X, coef, True))))\n",
    "print(\"Coeficientes:\\n\", coef.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotagem do Custo por Época"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG6FJREFUeJzt3Xl4nHW99/H3d5ZkkmZpkyZp2iQNbelG6ZpCsZWlbEVQ\nEZBFUJ7nqHXDC71Aj3o953h5PB718VE5qKgVVEQBkf3gBi1gqazpCjSFrnRLm3RNkzTr/J4/ZtKG\nUsgUZnLfM/N5XeSamXvuDp/fBf3019/ciznnEBGR9BHwOoCIiJwYFbeISJpRcYuIpBkVt4hImlFx\ni4ikGRW3iEiaUXGLiKQZFbeISJpRcYuIpJlQKj50+PDhrra2NhUfLSKSkZYvX77HOVeWyL4pKe7a\n2lrq6+tT8dEiIhnJzN5IdF8tlYiIpBkVt4hImlFxi4ikGRW3iEiaUXGLiKQZFbeISJpRcYuIpBnf\nFHdHdy+Llm7knxv2eB1FRMTXfFPcOcEAi5Zu4t6XtnkdRUTE13xT3IGAMX9iOU+/1kR3b9TrOCIi\nvuWb4gY4b1IFhzp6eGnzPq+jiIj4lq+Ke97Jw8kJBVjc0OR1FBER3/JVcefnhJg7tpTFDbtxznkd\nR0TEl3xV3ADnTa5g6752NjS1eh1FRMSXfFfc506sANByiYjI2/BdcY8ojjBlVBGLG3Z7HUVExJd8\nV9wQO7pkxdb97G3t9DqKiIjv+La4nYMn12m5RETkWL4s7lNGFjGiKMISrXOLiLyFL4vbzDh3UjlL\n1zfT0d3rdRwREV/xZXFDbLmkvauX5zft9TqKiIiv+La4zxhbSl44qOUSEZFj+La4I+Eg7z95OEt0\nFqWIyJv4trghtlyy82AHaxtbvI4iIuIbvi7ucyaWY4aWS0RE+vF1cZcV5jK9eqjOohQR6cfXxQ2x\n5ZI12w+yu6XD6ygiIr6QFsUNWi4REenj++IeX1FA1bA8LZeIiMT5vrjNjAtPGcGy9Xto6ej2Oo6I\niOcSKm4z22JmL5vZKjOrT3WoY108tZKu3iiL12rWLSJyIjPuc5xz051zdSlL8zZmVA9lZHGEv7zc\nONj/ahER3/H9UgnElksuOrWSpa9ruUREJNHidsBiM1tuZgtTGejtfOBULZeIiEDixT3POTcduAj4\ngpmdeewOZrbQzOrNrL65uTmpIUHLJSIifRIqbufcjvhjE/AQcNpx9lnknKtzztWVlZUlNyUQCGi5\nREQEEihuMxtiZoV9z4ELgFdSHex4tFwiIpLYjLsCWGZmq4EXgT875/6W2ljHp+USEREIDbSDc24T\nMG0Qsgyob7nkrufeoKWjm6JI2OtIIiKDLi0OB+xPyyUiku3SrrhnVA+lUsslIpLF0q64AwHjAzq6\nRESyWNoVN2i5RESyW1oWt5ZLRCSbpWVxBwLGRVO0XCIi2Sktixt0qVcRyV5pW9xaLhGRbJW2xd3/\n6JKD7VouEZHskbbFDfCRGaPo6o3y2Ms7vY4iIjJo0rq4TxlZxPiKAh5cscPrKCIigyati9vMuGxm\nFcvf2M+WPW1exxERGRRpXdwAl04fhRk8uFKzbhHJDmlf3COKI8wbN5wHV2wnGnVexxERSbm0L26A\ny2aOYvv+w9S/sd/rKCIiKZcRxX3hKSPIzwny4IrtXkcREUm5jCju/JwQC6aM4M9rGuno7vU6johI\nSmVEcQNcPrOKQ509PKFT4EUkw2VMcc8ZU0plcUTLJSKS8TKmuIMB49IZo1i6fg9Nhzq8jiMikjIZ\nU9wAl80YRW/U8egqnQIvIpkro4r75IpCplYV6xR4EcloGVXcEJt1r21soaGxxesoIiIpkXHF/cFp\nIwkFjId0CryIZKiMK+7SglzOnlDOQyt30NMb9TqOiEjSZVxxA1w+cxTNhzpZur7Z6ygiIkmXkcV9\n3uQKhhfkcvcLW72OIiKSdBlZ3OFggKtmV/HkuiZ2HDjsdRwRkaTKyOIGuHp2DQ7444uadYtIZsnY\n4q4uyees8WXc+9I2uvUlpYhkkIwtboBrTx9N06FOljQ0eR1FRCRpMrq4z5lQRmVxhLu1XCIiGSSj\nizsUDHDV7GqWvt7M1r3tXscREUmKhIvbzIJmttLMHktloGS7enYNwYBxz0uadYtIZjiRGfeNQEOq\ngqTKiOII504s576XttHVoy8pRST9JVTcZlYFXAzcnto4qfGx02vY29bF31/d5XUUEZH3LNEZ9y3A\nV4G0nLKeeXIZVcPydCaliGSEAYvbzC4BmpxzywfYb6GZ1ZtZfXOzv64REggYHzu9huc27WVjc6vX\ncURE3pNEZtxzgQ+Z2RbgXmC+mf3+2J2cc4ucc3XOubqysrIkx3zvPjqrmlDAuEezbhFJcwMWt3Pu\n6865KudcLXA18KRz7rqUJ0uyssJcLpwygvtXbKeju9frOCIi71pGH8d9rGtPr+FAezePrWn0OoqI\nyLt2QsXtnHvaOXdJqsKk2hljShlfUcAdyzbjnPM6jojIu5JVM24z41PzxtDQ2MKzG/d6HUdE5F3J\nquIG+ND0kQwvyOH2ZzZ5HUVE5F3JuuKOhIN84oxannqtmQ1Nh7yOIyJywrKuuCH2JWVuKMAdyzZ7\nHUVE5IRlZXGXFuRy2cwqHlixg72tnV7HERE5IVlZ3ACfnHcSXT1R7nr+Da+jiIickKwt7nHlBcyf\nWM5dz72hE3JEJK1kbXEDfOr9J7G3rYuHV+7wOoqISMKyurjPGFPK5MoibtcJOSKSRrK6uM2MT73/\nJDY0tfL06/66oqGIyNvJ6uIGuGTqSCqKcrnjGR0aKCLpIeuLOycU4Pr31bJswx7W7mzxOo6IyICy\nvrgBrj1tNPk5QRYt3eh1FBGRAam4geL8MNeeXsOjq3eyeU+b13FERN6Rijvu02eOIRwM8NMnN3gd\nRUTkHam448oLI1x7+mgeXrWDN/Zq1i0i/qXi7uczZ40hGDBue0pr3SLiXyrufiqKInzstBoeWLGd\nbfvavY4jInJcKu5jfOasMQTMuO1pzbpFxJ9U3MeoLM7jqtnV3L98GzsOHPY6jojIW6i4j+OzZ48F\n4OdP6wgTEfEfFfdxjBqax0frqrnvpe00HtSsW0T8RcX9Nj531liizvHLf+imwiLiLyrut1Fdks8V\ns6q4+8Wt7G7p8DqOiMgRKu538Pmzx9EbdfziHzrCRET8Q8X9DmpK87l85ij+8MJWtu/Xcd0i4g8q\n7gF8+fzxGPCjx1/3OoqICKDiHlBlcR7/Mu8kHlq1g1d2HPQ6joiIijsRnzt7LEPzwnz/b+u8jiIi\nouJORFEkzBfnn8wz6/ewVPemFBGPqbgTdN2c0dSU5PPdv66jN6o7wouId1TcCcoJBfjKhRNoaGzh\n4ZU7vI4jIllMxX0CLj61kmlVxfzw8dfo6O71Oo6IZKkBi9vMImb2opmtNrNXzexbgxHMjwIB42sX\nTWLnwQ5+++wWr+OISJZKZMbdCcx3zk0DpgMLzGxOamP51xljS5k/sZyfPbWB/W1dXscRkSw0YHG7\nmNb4y3D8J6u/nfvXBRNp6+zhZ0/psq8iMvgSWuM2s6CZrQKagCeccy+kNpa/TRhRyEdnVXPnc1vY\n1Nw64P4iIsmUUHE753qdc9OBKuA0M5ty7D5mttDM6s2svrk58491vunC8URCQb756Ks4l9V/ARGR\nQXZCR5U45w4ATwELjvPeIudcnXOurqysLFn5fKu8MMJNF4znmfV7+Osru7yOIyJZJJGjSsrMbGj8\neR5wPqBzv4mdlDO5soj/+J+1tHX2eB1HRLJEIjPuSuApM1sDvERsjfux1MZKD6FggG9fOoVdLR3c\numS913FEJEuEBtrBObcGmDEIWdLSrNHDuLKuijuWbebyWVWMryj0OpKIZDidOZkE/7pgIkNyQ/zb\nw6/oi0oRSTkVdxKUFuTy1QUTeGHzPh5ZtdPrOCKS4VTcSXL17BqmVRXzn39uoKWj2+s4IpLBVNxJ\nEgwY3750CnvbOvnxE7rNmYikjoo7iaZWDeXa02u489ktus2ZiKSMijvJvnLBRMoKc7n5T6vp7NGl\nX0Uk+VTcSVacH+a7l53Kul2H+MkSXYRKRJJPxZ0C8ydW8NFZVfz8HxtZve2A13FEJMOouFPk/1wy\nmfL4konuliMiyaTiTpHivDDfu3wq65tauWWxTocXkeRRcafQWePLuOa0ahYt3ciKrfu9jiMiGULF\nnWLf+MAkKovztGQiIkmj4k6xwkiY718+lU3Nbfzw8de8jiMiGUDFPQjmnTyc6+bUcPuyzby4eZ/X\ncUQkzam4B8nXL5pETUk+N967UneHF5H3RMU9SIbkhvjpNTPZ29rFzX9arcu/isi7puIeRKdWFfON\nD0xkybombn9ms9dxRCRNqbgH2fXvq2XBKSP4/t/W6RBBEXlXVNyDzMz4/hVTGVEc4Yt3r+Rgu67d\nLSInRsXtgeK8MD/92EyaDnVw8/1a7xaRE6Pi9sj06qF87aJJPLF2N7/55xav44hIGlFxe+hf5tZy\n/uQKvvvXBl1FUEQSpuL2kJnxgyumUl4Y4bO/X07ToQ6vI4lIGlBxe2xofg6LPjGLA+3dLPzdcl3P\nREQGpOL2gVNGFvPjq6axatsBvvbAGn1ZKSLvSMXtEwumVHLzBeN5eNVObnt6o9dxRMTHQl4HkKO+\ncM44Xt/dyg/+/honlxdwwSkjvI4kIj6kGbePmBn/94qpTKsq5kt/XMXanS1eRxIRH1Jx+0wkHORX\nn6ijKBLm07+rp/lQp9eRRMRnVNw+VF4U4fbr69jb1snCu+pp7+rxOpKI+IiK26emjCrmlqtmsHrb\nAT77+xV09US9jiQiPqHi9rEFU0bwvcumsvT1Zr583yp6ozpMUER0VInvXTm7mgOHu/ivv6yjKBLm\nvz4yBTPzOpaIeEjFnQYWnjmWA+3d3Pb0Roblh/nqgoleRxIRDw1Y3GZWDfwOqAAcsMg599+pDiZv\n9pULJ3DgcKy8i/PCfOassV5HEhGPJDLj7gFucs6tMLNCYLmZPeGcW5vibNKPmfHtD0+h5XA33/3r\nOorzwlx9Wo3XsUTEAwMWt3OuEWiMPz9kZg3AKEDFPciCAeNHV07nUEcPX3/oZQIB48q6aq9jicgg\nO6GjSsysFpgBvHCc9xaaWb2Z1Tc3NycnnbxFTijAL66bxbxxw/nq/Wu467ktXkcSkUGWcHGbWQHw\nAPAl59xbzsV2zi1yztU55+rKysqSmVGOkZcT5Pbr6zhvUgX/9sir/GrpJq8jicggSqi4zSxMrLT/\n4Jx7MLWRJBG5oSA/v24mF59ayXf+0sCtS9brcrAiWSKRo0oMuANocM79KPWRJFHhYID/vno6ueEA\nP3ridTq6e/nKhRN0nLdIhkvkqJK5wMeBl81sVXzbN5xzf0ldLElUKBjg/10xjUg4yG1Pb+Rwdy//\nfslklbdIBkvkqJJlgFrAxwIB4zuXTiE3FOA3/9zCvrYuvn/5VCLhoNfRRCQFdOZkhjAz/v2SyQwv\nyOUHf3+NHfsPs+gTdZQMyfE6mogkmS4ylUHMjC+cM46fXDODNTsO8pHb/snG5lavY4lIkqm4M9AH\np43knk/PobWjh8tue5bnNu71OpKIJJGKO0PNGj2Mhz4/l+EFOXzi1y/wwPLtXkcSkSRRcWewmtJ8\nHvzcXGbXlnDTn1bz3b800NOrGzKIpDsVd4Yrzg/z2/99GteeXsMvl27iY796gV0HO7yOJSLvgYo7\nC+SEAnznI6dyy1XTeWXnQS6+9RmWrd/jdSwReZdU3Fnk0hmjePSGuZQMyeHjv36BWxa/rtuhiaQh\nFXeWGVdeyCM3zOUj00dxy+L1/K/fvMie1k6vY4nICVBxZ6H8nBA/vHIa37vsVF7YvI8FtzzD46/u\n8jqWiCRIxZ2lzIyrT6vh0RvmUl6Yy8K7lvPlP67iQHuX19FEZAAq7iw3cUQRj9wwly+ddzL/s3on\nF/x4KUsadnsdS0TegYpbCAcDfOm88Tz8hdgXl5+8s56b/7Sag4e7vY4mIseh4pYjpowq5tEb5vHF\n+eN4aOUOLvjxP3hk1Q7doEHEZ1Tc8iY5oQA3XTCBhz7/PiqKItx47yquWvQ8DY1vuVudiHhExS3H\nNbVqKA9/fi7fu+xUNjS1cvGtz/DNR17hYLuWT0S8puKWtxUIxI48eeqms/n4nNHc9fwbnPPDp7nn\nxa265omIh1TcMqDi/DDf+vAUHvvi+xlbNoSvP/gyF96ylMfW7CSqMy9FBp2KWxI2eWQR933mDH5x\n3SyCAeOGu1dyyU+WsaRht77AFBlEKm45IWbGgikj+OuNZ3LLVdNp6+rhk3fWc/nPn+XZDbpwlchg\nsFTMlOrq6lx9fX3SP1f8p7s3yv3Lt3PrkvU0HuxgZs1QFp45hvMnjyAY0D2mRRJlZsudc3UJ7avi\nlmTo6O7lvvpt3P7MZrbua2d0aT6fmncSV8yqJi9Hd5sXGYiKWzzTG3X8/dVd/HLpJlZvO8Cw/DAf\nnzOa6+aMprwo4nU8Ed9ScYvnnHPUv7GfRUs3sbhhNwEzzp1YzjWn1XDm+DIto4gc40SKO5TqMJKd\nzIzZtSXMri1h85427n1xK/cv387ja3czsjjCR+uquXJ2NaOG5nkdVSTtaMYtg6arJ8riht3c8+JW\nlsWPQJk3bjgfnDaSC08ZQXFe2OOEIt7RUon43rZ97dxXv42HV+1g277D5AQDnDk+VuLnTqqgIFd/\nGZTsouKWtOGcY/X2gzy2eiePrWlkV0sHuaEA50wo59xJ5Zw9oZyywlyvY4qknIpb0lI06li+dT+P\nrd7J317dxe6WTsxiF7yaHy/yU0YWYaYvNiXzqLgl7TnnWNvYwpMNTTz5WhOrth3AOSgvzOV9Y0uZ\nM6aUM8aWUlOSryKXjKDiloyzp7WTf7zWzNOvN/Pcxr1H7kw/sjjCnLGlnDGmlLraEmpLVeSSnlTc\nktGcc2xsbuW5jXt5ftM+nt+0l71tsZscD80PM7166Jt+hubneJxYZGAqbskqzjnWN7Wycut+Vm49\nwKptB3ht9yH6/teuKclnUmUhkyqLmFRZxOTKIqqG5WlmLr6S1BNwzOzXwCVAk3NuynsNJ5JsZsb4\nikLGVxRy1ewaAFo7e1izPVbir+5ooaGxhcfX7j5S5oWREBMqChlbVsC48gLGlg9hbFkBVcPydVan\n+N6AM24zOxNoBX6XaHFrxi1+1N7Vw7pdh2hobGHtzhbW725lY3PrkWUWiN1zs7Y0n5qSIdSU5FNT\nkkd1ST41JflUl+QTCeuCWZIaSZ1xO+eWmlntew0l4rX8nBAza4Yxs2bYm7bvb+ti055WNja1sbG5\nlY3NbWzf386zG/fQ3tX7pn1LhuRQWRyhsjgv9jg0wsjiPMqLcikvzKWsIEJRXkjLMJJSOj1Nst6w\nITnMGlLCrNElb9runGNvWxdb97WzLf6z82AHjQcOs31/Oy9t2cfBw2+9eXJOMEBZYS5lhbkML8hh\nWH4OJUNiP8OG5FCSn8OwIWGK88IU5cUec0OayUviklbcZrYQWAhQU1OTrI8V8YyZMbwgl+EFuW+Z\npfdp6+yh8WAHzYc6aW7tpKmlg+bWztjrQ53sPNDBqztb2NvWRVfP299gORIOUBwv8YLcEIWRMAWR\nEIW5IQojIQpywwzJDZKfEzr6mBMkPzdEXjhIXjhIJCdw5HkoqJtbZbKkFbdzbhGwCGJr3Mn6XBE/\nG5IbYlx57AvOd+Kco72rl31tXexr62J/excHD3fTcribg8f8tHb2cKC9i23722nt6KG1s+ctSzYD\nCQeNSChIbjhIbihAbjhAbij+PBQgp99jTjD+GAoQDvb92JuehwLxx2CAUCD2XjBghINGMBDbFgzY\n0cegEbDY877HUMAIBIygxbYFAhx5/+i+EDiyjSPvBSz2B2nfY7bTUonIIDAzhuSGGJIborok/4R/\nfW/U0d4VK/C2eJHHCr2Hw11RDnf3cri7l87uXg539dLe3Utnd5SOnthjZ08vHfHHzp4ohzp62NsT\npas3SldP/Kc3SveRH0dv1L/zr2OL3IiVvMXL3gDs6Dbj6H72pm1gHH0NR/9gMDv6/pHX/T6H+L+j\n3wOlQ3K577NnpHz8iRwOeA9wNjDczLYD33TO3ZHqYCJyVDBgFEbCFEYG79K30aijOxor8Z54mfdE\no/T0Orp7o/REY+XeG3Xx57H3eqOO7qgj2ve+iz93sddR5+iNcmRbNP5+1HHkfec4sn9flqgDR+wx\nGnU4YvtFXexvNO6Y/foOmOv7vL5tjvj+jjdth773Ytvi/8S2xz/fHbOt79f0PSmMDM5cOJGjSq4Z\njCAi4i+BgJEbCKIr7PqPvsEQEUkzKm4RkTSj4hYRSTMqbhGRNKPiFhFJMypuEZE0o+IWEUkzKm4R\nkTSTkjvgmFkz8Ma7/OXDgT1JjJMuNO7sonFnl0TGPdo5V5bIh6WkuN8LM6tP9GLimUTjzi4ad3ZJ\n9ri1VCIikmZU3CIiacaPxb3I6wAe0bizi8adXZI6bt+tcYuIyDvz44xbRETegW+K28wWmNlrZrbB\nzL7mdZ5UMrNfm1mTmb3Sb1uJmT1hZuvjj8e/yWGaMrNqM3vKzNaa2atmdmN8e6aPO2JmL5rZ6vi4\nvxXfntHj7mNmQTNbaWaPxV9ny7i3mNnLZrbKzOrj25I2dl8Ut5kFgZ8BFwGTgWvMbLK3qVLqt8CC\nY7Z9DVjinDsZWBJ/nUl6gJucc5OBOcAX4v+NM33cncB859w0YDqwwMzmkPnj7nMj0NDvdbaMG+Ac\n59z0focBJm3svihu4DRgg3Nuk3OuC7gX+LDHmVLGObcU2HfM5g8Dd8af3wlcOqihUsw51+icWxF/\nfojYb+ZRZP64nXOuNf4yHP9xZPi4AcysCrgYuL3f5owf9ztI2tj9UtyjgG39Xm+Pb8smFc65xvjz\nXUCFl2FSycxqgRnAC2TBuOPLBauAJuAJ51xWjBu4BfgqEO23LRvGDbE/nBeb2XIzWxjflrSx625y\nPuScc2aWkYf7mFkB8ADwJedcS98dtSFzx+2c6wWmm9lQ4CEzm3LM+xk3bjO7BGhyzi03s7OPt08m\njrufec65HWZWDjxhZuv6v/lex+6XGfcOoLrf66r4tmyy28wqAeKPTR7nSTozCxMr7T845x6Mb874\ncfdxzh0AniL2/Uamj3su8CEz20Js6XO+mf2ezB83AM65HfHHJuAhYsvBSRu7X4r7JeBkMzvJzHKA\nq4FHPc402B4Fro8/vx54xMMsSWexqfUdQINz7kf93sr0cZfFZ9qYWR5wPrCODB+3c+7rzrkq51wt\nsd/PTzrnriPDxw1gZkPMrLDvOXAB8ApJHLtvTsAxsw8QWxMLAr92zn3H40gpY2b3AGcTu2LYbuCb\nwMPAfUANsSsrXumcO/YLzLRlZvOAZ4CXObrm+Q1i69yZPO6pxL6IChKbKN3nnPsPMyslg8fdX3yp\n5Gbn3CXZMG4zG0Nslg2x5ei7nXPfSebYfVPcIiKSGL8slYiISIJU3CIiaUbFLSKSZlTcIiJpRsUt\nIpJmVNwiImlGxS0ikmZU3CIiaeb/A9K/yZ4yvFm1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f25d831d3c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(errorHist)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Estimativa dos Coeficientes pelo Método dos Mínimos Quadrados (OLS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Métodos dos Mínimos Quadrados\n",
      "RMSE: 0.7504359153109991\n",
      "Coeficientes:\n",
      " [ 5.55089003  0.6814076  -1.90044063  0.03666973  5.31267873 -0.08333219\n",
      "  1.07130361 -0.12315714 -7.79524045  0.75497812  0.54306977  1.19954932]\n"
     ]
    }
   ],
   "source": [
    "X = np.append(np.ones([X.shape[0], 1]), X, axis=1)\n",
    "beta = np.dot(np.dot(np.linalg.pinv(np.dot(X.T,X)), X.T), y)\n",
    "\n",
    "print(\"Métodos dos Mínimos Quadrados\\nRMSE: {}\".format(RMSE(y - predict(X,beta))))\n",
    "print(\"Coeficientes:\\n\", beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comentários\n",
    "\n",
    "Obs.: Os dados desse dataset foram disponibilizados pela Universidade do Minho (Portugal) :P\n",
    "\n",
    "Primeiramente, é interessante notar que os dados de saída ($Y$), o atributo \"Quality\", possue apenas valores discretos e bastante baixos. Em contra-partida, os atributos de entrada ($X$) se apresentam em várias escalas, e podem ser bem maiores que os valores de saída. Por esse motivo, para manter a estabilidade do Stochastic Gradient Descent, é necessário realizar algum tipo de Feature Scaling para normalizar os dados de entrada, gerando assim um treinamento mais estável.\n",
    "\n",
    "No meu código, utilizei a classe MinMaxScaler do próprio Scikit-Learn para realizar essa normalização de forma rápida. O Min-Max Scaling consiste em, para cada atributo, subtrair todos os valores pelo menor valor e dividir isso pela diferença entre o maior e menor valor. Isso garante, então, que todos os dados serão dispostos no intervalo fechado [0, 1].\n",
    "\n",
    "No meu código, também, utilizei a notação matricial das operações entre os coeficientes ($\\beta$) e os dados de entrada ($X$). Isso permite uma computação mais rápida, com menos linhas de códigos, e ainda mantém todas as características originais do problema. Uma outra estivativa de coeficientes, utilizando o Método dos Mínimos Quadrados, também foi apresentada e mostrou resultados similares aos do Gradiente Descendente Estocástico."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
